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This paper outlines the process by which the Ontario Education Research 
Exchange (OERE), part of the Knowledge Network of Applied Education 
Research, developed and launched an online hub of education research summaries 
to facilitate greater use of research by stakeholders in the field of education. The 
project is an effort in knowledge mobilization funded by the Ontario Ministry of 
Education to help increase the use of research to inform policy and practice in 
Ontario. The paper begins with an outline of the background and history of the 
project. Next, the three main components of the project are outlined— 
collecting/writing the summaries and creating the inventory, putting together the 
peer review process, and creating the online hub for storing and sharing the 
summaries and facilitating the peer review process. This paper provides useful 
information that can be translated to similar projects with the goals of 
summarizing, storing, and/or sharing research with a broad audience. 


The purpose of this paper is to provide information on the development of an online 
repository of research summaries—the Ontario Education Research Exchange, an online hub 
hosting research summaries and facilitating community building between education stakeholders 
such as researchers, practitioners, policy makers, parents, and members of educational 
organizations across Ontario. 1 Although there is a call for sites such as this, particularly in the 
field of education, to mobilize research knowledge (Willinsky, 2001), there is very little 

1 To read the summaries visit the OERE website at http://oere.oise.utoronto.ca/ 
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information available about the processes involved in designing and creating such a repository. 
Further, the information available is largely from other fields and not entirely applicable to the 
field of education, as the nature of the publications, publication databases, and intended use of 
the research and repositories differ (Jadad, Carroll, Moore, & McQuay, 1996). 

To begin we provide the reader with the context the OERE project grew from: an 
intentional effort to build stronger linkages between research, policy, and practice, or knowledge 
mobilization (KM), in Ontario education—the Knowledge Network for Applied Education 
Research (KNAER). Next, the project plan is outlined along with the steps taken to launch the 
project which include creating the inventory, peer review process, and the creation of the online 
hub. Finally, the lessons that were learned through the process are described as well as how these 
lessons will be carried forward into the next phase of the project, as the project is currently only 
in phase one. The first phase of the project involved the creation of the project plan, designing 
the website that hosts the inventory and the peer review process, and beginning to acquire 
summaries from researchers and creating our own. The second phase of the project will involve 
actually launching the website and continuing to acquire and write summaries for the inventory. 

Background and History 

Increasing demands on limited resources have resulted in a culture of accountability in 
education and society in general (Mitton, Adair, McKenzie, Patten, & Perry, 2007), thus 
escalating the need for decision makers to use research evidence to support their policy 
decisions. As a result, there have been increased efforts to bridge what is referred to as the 
“knowledge gap” by incorporating evidence derived from research into policy and practice 
(Cooper, Levin, & Campbell, 2009). There has also been an increased interest in strengthening 
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the impact, use, and value of research in all sectors (Argote & Ingram, 2000; Court & Young, 

2003; Lavis, Robertson, Woodside, McLeod, & Abelson, 2003; Mitton et ah, 2007), including 

education (Cooper, Levin, & Campbell, 2009; Davies, Nutley, & Walter, 2005; Levin, 2011). 

Levin (2004) says this “increasing interest in research is driven by a more educated population 

and a growing awareness of the need to understand more fully the complex problems confronting 

us (Homer-Dixon, 2000).” Researchers, research funders, and professionals providing public 

services have become increasingly concerned about the productive use of research in schools and 

school systems (Cooper et al., 2009). However, strengthening the connections between research, 

policy, and practice is not easy. Sebba (2000) has identified several key obstacles, including 

developing commonly accepted criteria for evaluating the quality of education research across 

the broad range of methodologies used, improving access to research-based evidence, and getting 

education stakeholders to support and become actively involved in the process of bridging the 

gap (p. 10). As a result, increased efforts are taking place to build strong linkages between 

research, policy, and practice in education. The OERE team uses the term knowledge 

mobilization (KM), coined by the Social Sciences and Humanities Research Council (SSHRC), 

to refer to these efforts. SSHRC defines KM as 

moving knowledge into active service for the broadest possible common good. 

Here knowledge is understood to mean any or all of (1) findings from specific 
social sciences and humanities research, (2) the accumulated knowledge and 
experience of social sciences and humanities researchers, and (3) the accumulated 
knowledge and experience of stakeholders concerned with social, cultural, 
economic and related issues. (SSHRC, 2008, para 5) 

We prefer the term KM to the many others being used to describe this work (such as knowledge 

brokering) - as, unlike them, it emphasises the multi-dimensional, purposeful, and interactive 


2 For a synthesized list of the various terms and definitions currently being used please see 
www.oise.utoronto.ca/rspe 
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nature of these efforts rather than a more linear movement from research to practice (Levin 2008; 
Sa, Li, & Faubert, 2010). The OERE project, and indeed the greater KNAER initiative was 
conceived with an explicit KM mandate, and the term KM best encapsulates what we ultimately 
hope to accomplish with the OERE, by working to make Ontario-based education research more 
accessible to stakeholders in the Ontario education sector. 

It is important to acknowledge that efforts to strengthen the use of research evidence in 
education are not without controversy, including concerns about the degree to which research 
can guide practice, the amount and quality of research needed to derive conclusions, 
appropriateness of research topics, and other controversies. More specifically, there are 
limitations to the OERE approach to KM. Studies suggest that passive forms of dissemination, 
such as publishing research summaries, are less effective than more active forms such as face to 
face networking (Cooper & Levin, 2010; Nutley, Walter, & Davies, 2007). Cooper (2012), for 
example, notes that research summaries tend to be less effective because they often fail to 
include actionable messages on how to incorporate the findings into practice. Furthermore, data 
shows that passively posting research products online is unlikely to increase research uptake 
(Edelstein, Shah, & Levin, 2012). The OERE team was aware of these limitations and did our 
best to address these challenges throughout the process as described in the following sections of 
this paper. However, we do recognize that any project of this nature will inevitably have 
imperfections. 

The KNAER functions as a collaborative partnership between the Ontario Ministry of 
Education, the University of Toronto, and the University of Western Ontario, and works to 
support the Ministry of Education’s “commitment to develop and implement policies, programs, 
and practices that are evidence-based, research-infonned, and connected to provincial education 
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goals” (KNAER, 2011). The KNAER’s Planning and Implementation Committee (PIC) 
identified three projects to be developed and managed by the KNAER host institutions including 
the Ontario Education Research Exchange (OERE), one of the projects managed by the Ontario 
Institute for Studies in Education, University of Toronto (OISE), under the direction of Dr. 
Levin, professor and Canadian Research Chair in Educational Leadership, and director of the 
KNAER. The authors of this paper have all actively participated in the development of the 
OERE. 


The OERE 

As part of his role as Director of the KNAER, Dr. Levin was part of the initial 
development of an Inventory of significant recent Ontario research in education. The goal was 
to identify research in education conducted in Ontario in the last five to seven years through a) 
an online search of web pages; b) a search of SSHRC records; c) contacting university 
department heads, deans, VPs, and others for nominations; and d) a broad public call. Ultimately, 
research within the inventory is to be made available on the online hub for educators throughout 
the Ontario education system. An initial project budget of $50,000 was granted to achieve these 
goals, primarily for staff time and materials. It is from this initial vision that the OERE was born. 

Although some research is available on the World Wide Web through search engines 
such as Google, the OERE project is unique for five main reasons. First, the OERE focuses on 
developing and housing plain language summaries two to four pages in length that can be read 
by stakeholders such as practicing educators who may not have time to read longer research 
articles. Second, the OERE aims to develop plain language summaries that avoid the type of 
technical jargon often found in articles published within research journals. Third, the project 
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focuses on collecting and housing research summaries conducted in Ontario. Research conducted 
in Ontario is difficult to locate through online searches (and is not often grouped together in one 
place). Educators also strongly prefer research done in their local contexts as education is 
provincially mandated; therefore policies and programs differ from province to province. Fourth, 
the OERE’s searchable website is available free of charge on the Internet, and a strong effort was 
made to include graduate student and other unpublished work not regularly found through 
standard online searches. Many full-length research articles are not part of open access journals, 
limiting stakeholders to access them through places such as university websites (for which they 
would need to be students or faculty to gain access) or through other organizations that have 
access to journal websites. Finally, the OERE’s online hub is intended to create a greater sense of 
community within the Ontario research community and help develop networks and connections 
between all stakeholders. A main objective of the broader Ontario Education Research Strategy 
is to build a community with a sense of common research enterprise across the province which 
differs from the previously disconnected efforts of researchers and practitioners (Campbell & 
Fulford, 2009). We hope that the OERE engenders a stronger sense of self-worth and efficacy in 
the Ontario education community. 

The original project plan, formulated in January 2011, determined that the inventory 
would be called the Ontario Education Research Exchange (OERE). Our vision of the OERE 
was “an online hub that would facilitate the exchange of advanced applied knowledge between 
researchers, practitioners, policy makers and educational organizations across Ontario” (OERE, 
2011). The project had three main components: 

1. creating the inventory, collecting research, and writing/collecting summaries for 
the online hub; 
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2. a peer review process, involving both practitioners and academics to help ensure 
the site highlighted high quality plain language research summaries with clear 
implications for practice; 

3. an online hub that would: 

a. act as a database to file and store the summaries; 

b. facilitate an online peer review process; and 

c. assist with a larger online knowledge mobilization campaign to encourage 
further participation with the ultimate hope that the project would 

”... create linkages between stakeholders and regions working to address 
similar educational issues in order to create innovative, evidence-based 
solutions.” (OERE, 2011) 


This was an ambitious plan. The OERE concept was announced publically during the Ontario 
Education Research Symposium (OERS) in February 2011, as this event included members from 
each of our target audiences, which include researchers, practitioners, policy makers, and 
educational organizations. 

We had initially hoped that many researchers would contribute their own research 
summaries (or “abstracts” as we called them in the early days) to the OERE, summaries that 
would then be submitted to our peer review panel before being published online (see Appendix: 
Diagram of Original Project Flowchart). In order to encourage researchers to submit their work 
we contacted the deans of research at various faculties of education across Ontario and asked 
them to promote the project to their faculty and graduate students. We also contacted everyone 
who had attended OERS in order to further promote the project. However, after receiving little 
response, we took a more personalized approach over the summer and actively contacted 
researchers to request that they submit a summary of their work to the OERE. The lack of 
response, one might add, is consistent with the generally weak knowledge mobilization activities 
of faculties of education in Canada and elsewhere (Sa, Li, & Faubert, 2010). 
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Once the vision for the project was created, the project manager consulted with other 
organizations working on similar projects to develop a strategy for moving forward. A six-month 
plan was put forward that would see the creation of a team that would work together over the 
summer to identify appropriate research for the project, as well as solicit and support authors 
with writing plain language summaries. It was our hope that within a five month period, five 
team members working part time would collectively gather/produce over 350 research 
summaries, consult with stakeholders to develop an online peer review process, establish a peer 
review panel consisting of practitioners and researchers, and develop the OERE website, online 
database, and online peer review process. Our intention was to launch the OERE to the wider 
education community in September 2011 so that they could begin to use the database and the real 
KM work could begin. 

Creating the Inventory 

The following section provides an overview of the steps included in creating the 
inventory that would be posted on the online hub—collecting the research and writing or 
collecting summaries of this research. 

Inclusion and exclusion criteria. It was decided that items would be included in the 
project if they presented findings from studies that met the following criteria: 

■ if they were empirical with a significant sample size (25 or greater); 

■ if they were Ontario-based, or, at least part of the data had been collected from 

Ontario; 

■ if they were recent (2000-present); 

■ if they included methodology used, findings, and a discussion; 
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■ if they had practical relevance to Ontario Education. We were looking for items 
that would be useful for practitioners and other stakeholders in Ontario in their 
educational involvement. This includes things such as strategies and tools for 
educators or educational leaders, policy suggestions or implications for school 
boards and the ministry of education, or information for parents about how to help 
their children succeed in school. 

Journal articles from academic and other sources were one of the first places we looked; 
however, dissertations were also included as they also go through an external review process. 
Items meeting inclusion criteria were excluded if they had not been subject to some form of 
external review or evaluation and if the researchers were unwilling to respond to requests for 
information about their work. 

Inclusion-exclusion criteria were chosen in an attempt to stock the online hub with 
research summaries that were empirically based, methodologically sound, and relevant to 
education stakeholders such as researchers, practitioners, policy writers/analysts, and parents. A 
sample size of 25 was deemed sufficient to reach data saturation. The time frame was chosen to 
ensure research included in the online hub would be relevant to practitioners. However, the 
online hub will not necessarily exclude research that does not meet the inclusion-exclusion 
criteria, as researchers are able to submit summaries of their own to the peer review process. If 
the summaries pass the review process they will be posted on the hub. The inclusion-exclusion 
criteria listed above were what was used for the initial search conducted by the research team. It 
should be noted that it is important that research be conducted either partially or entirely in the 
Ontario context as the project is intended to infonn the work of Ontario education stakeholders 
(and is funded by the Ministry of Education for that particular purpose). 
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Searches. The searches were guided by the inclusion and exclusion criteria and a list of 
themes highlighted as priority areas by the Ministry of Education : teaching and learning; equity, 
transitions, and engagement. Questions the ministry hoped to have answered about teaching and 
learning included, what are the most effective strategies for improving early reading 
achievement, and to what extent are research based instructional strategies being implemented in 
junior mathematics? The theme of equity included questions such as, what is the relationship 
between SES and student access to resources, wellbeing and achievement, and what are the 
achievement patterns and trajectories among different sub-groups of students? The ministry 
hoped to answer questions about transitions, such as, how can we most effectively track long¬ 
term outcomes for students who leave the secondary system, and how can we facilitate 
successful K-12 and post-secondary transitions for students (including those with special 
needs)? The ministry hoped to find answers to questions about engagement, such as, what is the 
relationship between parent engagement, student engagement, student wellbeing, and student 
achievement, and what do we know about the relationship between student engagement and 
student outcomes? 

The research team divided up the priority research areas so that each member initially 
focused on finding articles from one area. Beyond these four ministry priority research areas, 
broad searches were also conducted for research pertaining to education in Ontario in general, 
using a series of search strings, including: “Ontario education study,” “research Ontario 
education,” “study school Ontario,” etc. Search strings were saved and compared by team 
members to ensure as broad a search as possible was conducted, within the parameters. Searches 

3 For the fully explained list of priority themes please see http://www.edu.gov.on.ca/eng/research/EDUPriority 
_ThemesKNAER.pdf 
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were conducted using multiple search engines including Google Scholar, JStor, ProQuest, ERIC, 
CBCA Education, and the library databases from OISE and Western. Over 100 articles were 
found through the initial search. 

Author communication. Once the initial batch of articles and reports were found, an 
attempt was made to find contact information for the researchers who wrote each article or report 
using search engines and professional contacts or connections. When contact information was 
found, an email was sent informing them of the OERE and inviting them to submit a summary of 
their work, collaborate with the OERE team, or direct the OERE team to write a summary for 
them. In some cases the contact information for the first author was not available; in this case if 
the contact infonnation for the second author was available, they were contacted and asked to 
consult with the first author. The status of all communications with researchers was logged, 
including information about whether or not we had received a response from the researcher: if 
the researcher wanted to write a summary, collaborate, or have OERE write a summary for them. 
Infonnation about the status of the research summaries was recorded in the same chart, which 
included whether or not the researcher or OERE was in the process of writing or editing a draft, 
if the researcher had given approval on the final version of the summary, and if the summary was 
ready for peer review. 

Storing data and summaries. Multiple programs were used to store and organize articles 
and reports, infonnation about the status of different tasks (searchers, author communication, and 
summary writing), the summaries, and the information required for the peer review process and 
website. Refworks, an online bibliographic management program, was used to store the articles 
and reports that were found and their bibliographic information. Information about search strings 
were stored in Excel spread sheets and shared amongst the group. PBworks was used to store 
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information about the status of communication between researchers and the OERE team, as well 
as the status of the summaries (if they were being written, edited, or ready for peer review, etc.). 
PBworks is an online document-sharing program that allows multiple users to work on the same 
documents together. Microsoft Publisher, a desktop publishing application that aids in the 
creation of document templates, was used to create the templates for the summaries. All of the 
summaries were stored, shared, and worked on collaboratively using Drop Box, a free online 
document-sharing program. All of these programs were intended to facilitate the storing, sharing, 
and collaboration that were required for maintaining, organizing, and working on project items 
and related data. 

Writing strategies. In April 2011 we partnered with the Research Impact team at York 
University to benefit from their expertise in writing plain language summaries, and to take 
advantage of the opportunity to share our work more broadly through their “Research Snapshot” 
database. Four graduate students were hired to work on the project in the summer and received 
training in plain language writing through Research Impact. Over the course of a day at York 
University, the members of the OERE writing team were given an introduction and training in 
plain language writing. Although it was not intended to be comprehensive training on plain 
language writing, it was enough for us to understand the philosophy behind the movement and 
get an inkling of process involved in writing research summaries. 

The OERE project was conceived with the explicit goal of "translating" research articles 
into plain language. The vast majority of research articles, written and published in print journals 
or online, and conference proceedings use technical language specific to the field or discipline in 
question. Social sciences and the field of education are no exception. In recent years, government 
directives in a number of industrialized nations to make public communications clearer and more 
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accessible have led to renewed interest in plain language writing (Kimble, 1994, 1995; Rudd et 
al., 2004). Technical writing is often cited as a barrier to using and understanding research 
literature and subject specific literature by students (Shkedi, 1998; Wellington & Osborne, 2001; 
Willinsky, 2001). More recently, the plain language writing movement has been making an 
impact, convincing scholars of its importance through its use in things such as knowledge 
mobilization initiatives (Willinsky, 2001). This becomes especially important where education 
research is concerned, as education stakeholders such as parents, teachers, school administrators, 
and students stand to benefit from its findings (Willinsky, 2001). This makes the availability of 
plain language versions of education research articles an urgent need. 

Although the term "plain language" has multiple meanings, depending on the discipline 
or industry in question, the intent of the movement is to communicate in ways that ensure the 
broadest possible audiences. As described by the Plain Language Association International, plain 
language "is communication designed to meet the needs of the intended audience, so people can 
understand information, that is important to their lives" (Stephens, 2000). A history of the plain 
language movement finds multiple origins around the English-speaking world over the last fifty 
years. Mazur (2000) traces the plain language movement back to government initiatives in the 
early 1970’s under President Nixon, with the aim of communicating policy more effectively to 
the public. McBeth (2002) takes a more international view, providing the Canadian, New 
Zealand, Australian, British, and South African contexts of the movement. McBeth credits Phil 
Knight of the Plain Language Society in Vancouver as an early leader of the plain language 
movement in Canada (2002, p. 11). Kimble positions plain language as having "to do with clear 
and effective communication—nothing more or less" (1995, p. 2). Steinberg (1991) looks at 
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plain language from the perspective of the reader, as reflecting their interests as opposed to those 
of the writer. 

During the writing process, the OERE writing team was respectful of the fact that 
research summaries would be written in plain language with the goal of increasing readability 
and summarizing key findings, the goal was not to "dumb down" the articles. But this proved to 
be a challenge, as the team had to convey the nuance of the findings while avoiding modifiers 
and qualifiers (e.g ., possibly, indicates that, potentially, etc.). Primarily schooled in academic 
writing, the team found it difficult to adjust to the plain language writing style. The writing 
team's original approach, of asking authors to write summaries of their own articles in plain 
language, had mixed results as some authors tended to revert to technical language, only in a 
condensed fonn. The research team addressed this issue by modifying its approach to writing a 
summary and then sending it to the authors for edits or approval. Even though the team was not 
intensively or formally trained in plain language writing, the more summaries of research articles 
they wrote, the more their skills in plain language writing improved, to the point where by the 
end of the project some research team members summarized several articles a day, most of 
which the authors approved after only minimal edits. 

Each summary included five sections. The first, What this research is about, tells the 
reader the purpose of the research, or what phenomenon or reason the researchers had for 
conducting the study. The second section, What did the researchers do, explains the methods 
used for gathering the data. What did the researchers find, the third section, explains the key 
findings of the study. The fourth, How can you use this research, explains how the key findings 
could be applied to thinking and practice. Finally, What you need to know is a very short 
synopsis of the summary, explaining the key points of the summary in just a few sentences. 
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Peer Review Strategy 

The purpose of the OERE peer review was to have all research summaries evaluated 
through a multi-stakeholder review process to ensure they met research quality standards and had 
practical implications for Ontario educators; this included those summaries prepared by the 
OERE writing team and in conjunction with researchers. Similar to the academic peer review 
process, summaries are reviewed and evaluated, some given instructions regarding revisions that 
could be made while other summaries are rejected. Unlike academic peer review, we did not ask 
for a careful assessment of the original research, but whether the summary included the 
information necessary to be of reasonable quality and utility, which includes: purpose, methods, 
findings, and how the findings can be used. Research shows that education practitioners are more 
likely to value research that is done in their local contexts or in contexts that are similar to their 
own (Nelson, Leffler, & Hansen, 2009). Therefore, unlike most review processes, we also 
wanted every summary to be validated by practitioners as having practical relevance to their 
schools. After reviewing other peer review process documents from various academic journals 
and different rubrics used by policy makers and practitioners to evaluate education research, we 
determined that we would develop a two-prong peer review process with separate evaluation 
rubrics for academics, practitioners, and administrators. Draft rubrics were created and sent to 
the members of the PIC and a group of interested scholars and practitioners for feedback. Once 
the review process was decided, the web design team created the online peer review processes. 
The peer review process was designed with the lives of busy researchers, practitioners, policy 
makers, and other educational stakeholders in mind. 
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Creation of the Online Hub 

According to the original OERE development plan, creating the OERE online hub had 
three purposes: 1) it would act as a database where summaries that had cleared peer review could 
be stored and accessed by education stakeholders; 2) it would facilitate an online peer review 
process of these summaries; and 3) it would be used as part of a larger knowledge mobilization 
strategy intended to create a space for dialogue and exchange between education stakeholders. 

We followed OISE guidelines for finding an appropriate consultant to help us build the 
site, which included approaching multiple firms to discuss the project and get an official quote. 
Ultimately we chose to work with a small company that seemed to best grasp the concept and 
provided a reasonable quote and timeline to complete the website. We then worked with the 
company in order to create the detailed specifications upon which the website would be based. 
The work on the website continued throughout the summer and fall of 2011. Due to unforeseen 
complications at their end, the design team was not able to meet their original timelines and the 
site was not completed and tested until late fall 2011. 

Lessons Learned 

In the following section we report on key lessons learned from the three main 
components of the OERE project - creating the inventory, creation of a peer review panel, and 
development of the OERE online hub, with particular attention to the findings we feel may be 
useful for people involved in similar projects. 
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Creation of the Inventory 

From May to August 2011, the OERE summer writing team identified 100 research 
studies that met the inclusion-exclusion criteria. The team attempted to contact at least one 
researcher involved in each of the studies by email. By the end of August we had received 
responses from 54 researchers—some of whom were involved in more than one of the identified 
studies. Some of these researchers agreed to write and submit summaries to the OERE on their 
own, although most were written as part of a collaborative back-and-forth writing process 
between the OERE team and the researcher. Many of the summaries were drafted by the OERE 
team and then submitted to the researcher for feedback. In total, close to 70 research summaries 
were completed or in draft form by the end of August 2011. This number was far short of the 
summary target of 350 because several unexpected factors impeded our progress: locating 
Ontario-based education research studies with a sample size of over 25 was more difficult than 
first imagined. Technical challenges also hindered our progress as we sought to identify, learn, 
and use new technology that was capable of meeting our growing data management needs and, at 
the same time, allowed all our team members access regardless of time of day and location (a 
necessity since many of our team members worked at different times and in different locations). 
Further, our team found that identifying accurate contact infonnation for researchers was 
difficult; if contact information was not included in articles, internet searches were conducted. 
Lastly, two findings that were particularly surprising were the lack of response from researchers 
about developing summaries, and the challenges of the clear language writing process itself. 

Many researchers did not respond when contacted; several stopped communicating 
during the writing process, and few researchers submitted summaries themselves using our 
online system. Although the OERE had originally intended for all (or at least the majority) of 
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researchers that we contacted to develop and submit research summaries via our online system, 
by the end of August 2011 only eight researchers had taken the opportunity to do so. The 
remaining 60 summaries were either written in collaboration with the OERE writing team or 
were developed entirely by the OERE writing team and shared with the researcher after a full 
draft of the summary had been developed. In fact, writing our own full summaries and then 
submitting them to researchers for feedback was found to be the most efficient method of 
producing summaries, as it eliminated the lag time involved in communicating with busy 
researchers and the lack of response from some researchers. 

The clear language writing process also proved challenging for researchers and the 
writing team. Both groups were accustomed to writing for an academic audience, it was difficult 
to write in a way that adequately conveyed the theoretical underpinnings of research and 
presented the research findings while, at the same time, remained free of jargon and overly 
complex language. While we were generally able to find a balance between technical or 
academic writing and plain language, this remains a contentious issue, as differences of opinion 
remain regarding the difference or similarity and value of academic and plain language. 

Creation of a Peer Review Panel and Process 
In developing this peer review panel, the OERE found 

■ the number of peer reviewers required to conduct an efficient peer review process 
was larger than anticipated; 

■ timelines and regular follow-up contact with reviewers was necessary to maintain 
a timely review process; and 

■ managing data, tracking summaries through the review process and 
communicating with large numbers of reviewers all benefitted from systematic 
strategies. 
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Throughout the summer and into the fall of 2011, the OERE recruited a small number of 
academics and practitioners to join our peer review panel. By January 2012, the OERE had a list 
of 15 individuals and we had 30 summaries ready for review. Although we originally felt 15 
people was a suitable number to begin our peer review process, we found early on that it was not 
nearly enough to conduct an efficient review. Four issues stood out. First, peer reviewers were 
assigned reviews according to their area of interest and expertise; there was not always a match 
between our reviewers’ areas of expertise and the research to be reviewed. Second, this problem 
was compounded by the fact that we required one practitioner and one researcher with an 
expertise and interest in the topic area to review every summary. Third, the availability of 
reviewers was not always guaranteed. Even if we found two reviewers for a summary, they were 
not always available at the time that we required. Fourth, we found that we had originally 
recruited more academics and we needed more experienced classroom teachers. For these 
reasons, we ran a second recruitment for reviewers by putting a notice out via OISE listservs and 
sending them to personal contacts within schools (some of whom agreed to send the OERE 
notice out to other teachers via school listservs). In the next few months we received a surprising 
number of responses, especially from teachers who expressed interest and enthusiasm in the 
goals of the OERE project. By March 2012 we had increased our peer review panel to 60 
reviewers and by April we had reached a total of 80 peer reviewers—approximately half teachers 
and half academics. 

Lastly, the OERE found developing systems for managing data and communicating with 
reviewers was essential to an efficient peer review process. Although our peer reviewers 
volunteered their time, we found it necessary to provide reviewers with a clear time frame to 
complete reviews. Our team found that a 2-week time period provided some flexibility to 
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reviewers, while maintaining a timely review process. We also found that regular follow-up 
emails to reviewers just prior to their 2-week deadline increased the response rate and 
encouraged reviewers to contact us if they were unable to complete a review. We found that 
maintaining up-to-date track records of summaries was useful. This included maintaining 
information on who was assigned to each summary, when a reviewer was assigned a summary, 
when the summary was due, and whether we had received one or both reviews of a summary. 
We found Excel spreadsheets and the use of colour-coding particularly helpful. 

Similarly, while facilitating the peer review process we found it useful to develop 
systems such as providing all new reviewers a copy of a “How to Guide” on the OERE peer 
review process, which included answers to common questions. We created standard email 
response templates to common peer reviewer questions and templates for common follow-up 
emails that were sent to reviewers. When we had recruited 60 reviewers in March we also found 
it useful to hire one person part-time whose sole responsibility was helping with administrative 
tasks, some of which are described above. 

Development of the Online Hub 

Technical delays were a regular occurrence when working with the design company to 
develop and launch the website. Although the OERE website and peer review process were 
intended to be launched in August 2011, we were still waiting to launch the site in November 
2011 because of technical delays. As the OERE’s online peer review process was linked to the 
development of the OERE website, delays in the development of the site postponed the 
possibility of submitting summaries to our peer review panel. When the OERE website was 
finally launched in January 2012, the project was four or five months behind schedule. 
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Numerous technical issues with the website were also identified by our own team during the 
testing of the site prior to launch, and by our peer review panel post launch. A common problem 
concerned compatibility issues between our online peer review system and the computers and 
operating systems used by peer reviewers, particularly older versions of computers and operating 
systems. 


Discussion—Carrying the Lessons Learned into the Next Phase 

At the time this article was written, the OERE team was preparing to launch a second 
summer writing team to collect further research and develop plain language summaries to our 
website. In this section we share how we expected to use our findings to improve our writing 
process, and offer suggestions for those interested in developing similar projects that may 
involve developing a peer review panel and an online site capable of storing research summaries 
and facilitating a review process. 

Improving the Research Collection and Summary Writing Process 

The restrictiveness of our inclusion-exclusion criteria was an initial barrier for us. Fewer 
summaries were found than initially anticipated. Jadad, Carroll, Moore, & McQuay (1996) had a 
similar problem when creating their database (although their team searched journals by hand and 
ours used online search engines), they found it helpful to provide those doing searches with a list 
of journals and other publication locations that would be likely to have the kinds of publications 
they were looking for, which we will try in the future. Another challenge was persuading 
education researchers to submit their own research summaries to the OERE or encouraging 
researchers to work with the OERE writing team to develop summaries. However, since our 
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team found it more efficient last summer (2011) to develop summaries ourselves and request 
researcher feedback only on fully developed draft summaries, we plan to use this method in the 
future. Our team has also learned from managing the peer review process that clearly defined 
timelines help people meet deadlines and keep the project moving forward. For this reason, our 
2012 summer writing team will request that researchers provide feedback on draft summaries 
within a 2-week period before they are submitted to our peer review panel. 

The plain language training we provide our summer writing team is something we are 
also working to improve. We found that the initial 1-day training we provided did not address 
some of our unique needs such as summarizing statistical analysis and results in plain language. 
Therefore, follow-up training would have been useful at the beginning of the summer when we 
were first becoming accustomed to the plain-language writing process. In summer (2012) we 
worked with York University to develop ongoing training sessions that are tailored to meet our 
specific needs. Another area we are working to improve is our data management strategies 
during the collection of research phase. In 2012 we aim to reduce the number of different 
technologies we use in order to decrease the amount of time our team spends learning new 
technology. 

Although we now have an efficient peer review process and a fully functioning website, 
this took much trial and error. We offer some suggestions below for others about to avoid some 
of the challenges we faced along the way: 

■ When developing a website and peer review process, we recommend that these 
two systems be developed separately from one another to avoid delays on one part 
of the project from influencing another part. 

■ Identify user-friendly technology that meets your project’s needs and use fewer 
technological devices whenever possible to reduce time spent on learning new 
systems. 
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■ When working with contractors, such as with website developers, make every 
effort to check their track record and references to minimize technological hold¬ 
ups and related issues. 

■ If outside training is required for your team, it may be more useful to arrange 
shorter ongoing training sessions rather than one training session at the beginning, 
as this will allow your team to discover questions they may not have had the day 
of training, and will allow your team to receive feedback development. 

In a summary report of 23 research repositories put together by CAIRSS (Council of 
Australian University Librarians Australian Institutional Repository Support Service, 2009) 
whose primary function is to offer support for repository managers in the higher education sector 
in Australia and New Zealand, they listed the following things that were difficult to solve (which 
are very similar to our findings): 

■ academic engagement and recognition of repository and open access benefits, 

■ non-availability of author manuscripts. 

They provided the following advice to others in their attempts to set up a repository: 

■ Make sure you have a strong relationship with your Research Office. 

■ Ensure you have good IT support. 

■ Identify and utilise key research champions to populate and promote the repository. 

The CAIRSS report also identified that lack of information and support available for 
setting up a research repository. Many of the case studies described how repository managers 
often felt as though their work was based on the trial and error method rather than based on any 
empirical knowledge regarding best practices, particularly given the unique nature of most 
repositories. Clearly we were not alone in our struggles to come up with a good model. We hope 
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the lessons we learned and outlined in this article will be of benefit to others hoping to create 
similar inventory or repository systems. 


Conclusion 

This paper has outlined the processes involved in the development of the Ontario 
Education Research Exchange (OERE), an online hub intended to facilitate the exchange of 
advanced applied research knowledge (from empirical studies conducted in or regarding Ontario) 
between researchers, practitioners, policy makers, and educational organizations across Ontario. 
The overall process involved seven main steps: 

■ creating a project plan, 

■ hiring and training a team of research assistants to collect and summarize research 
and liaise with researchers, 

■ finding the articles to be summarized, 

■ communicating with authors, 

■ writing summaries, 

■ creating the online hub, 

■ creating the peer review process and finding reviewers. 

Daunting human and technical challenges slowed the process considerably, putting the 
team four to five months behind the original schedule. Regardless of the challenges outlined in 
this paper the project was an overall success in terms of its larger goal of knowledge 
mobilization, with over 120 summaries currently uploaded on the OERE website for use by 
educational stakeholders. Knowledge learned from this process will be applied to the second 
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phase of this project as it moves forward and may be informative for others attempting similar 
knowledge mobilization projects. 
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Appendix: 

Diagram of Original Project Flowchart 


Researcher \ 
completes J 
online abstract J 




Researcher is 
informed 


■J 



Peer review 
evaluates 
abstract using 
online process 


Ideally, this peer review process would involve a large panel of people 
grouped by interest and expertise. The abstract would be sent to the 
appropriate group (ideally via a message in their inbox providing them with 
a link to the abstract and review form) and those who are interested and have 
the time will complete the review using an online form or survey tool. 



Abstract is 
accepted with 
revisions 


Researcher is \ 
► :> informed and j 

makes revision J 


Researcher 

informed 


'> 


Abstract 
is rejected 


Abstract is 
accepted 


Researcher 

informed 


V 



29 



















